1. Field of the Invention
The present invention relates to a luminance signal generation apparatus, a luminance signal generation method, an image pickup apparatus, and a storage medium. More particularly, the present invention relates to an apparatus and a method for generating a luminance signal from a color signal output from an image pickup device having color filters of Bayer pattern, an image pickup apparatus having the luminance signal generation apparatus, and a storage medium storing a program for executing the luminance signal generation method.
2. Description of the Related Art
To generate a color image by using an image pickup device (such as a CCD image sensor or a CMOS image sensor) capable of detecting an amount of light, light is generally made to pass through color filters and then incident to the image pickup device.
The color filters are various in terms of type of color and in terms of arrangement for color allocation to respective pixels. As the type of color, primary colors (red, green, and blue) or complementary colors (cyan, magenta, and yellow) are widely used. As the color arrangement, a Bayer pattern is widely used.
FIG. 21 shows one unit of a primary color Bayer pattern, in which symbol R represents red, G1 and G2 represent green, and B represents blue. The primary color Bayer pattern has units (one of which is shown in FIG. 21) whose number corresponds to the number of pixels provided in the image pickup device.
Conventionally, as methods for generating a luminance signal from a color signal output from a solid-state image pickup device, there have been known two methods, i.e., out-of-green method and SWY method.
In the out-of-green method, red, green, and blue color signals are processed independently of one another when the luminance signal is generated by using color filters of the primary color Bayer pattern shown in FIG. 21. With the out-of-green method, the luminance is determined mainly from the green signal.
To process the red signal, zero data is inserted into pixels (other than red pixels) of a RAW signal obtained by digitizing the color signal output from the image pickup device. Low-pass filter processing is then performed to limit vertical and horizontal bandwidths. Green and blue signals are processed in the same manner as the red signal, whereby the red, green and blue signals are provided to respective pixels. The luminance signal Y is determined according to, e.g., formula (1) given below.Y=0.3R+0.59G+0.11B  (1)
Then, a high-frequency emphasis signal is generated only from the green signal and added to the luminance signal Y, thereby obtaining an OG signal.
Alternatively, the green signal subjected to low-pass filter processing can be used as the luminance signal Y. In that case, the OG signal is generated only from the green signal.
In the SWY method, RGB pixels are all used when the luminance signal is generated by using color filters of primary color Bayer pattern shown in FIG. 21. In other words, the RAW signal obtained by digitizing the color signal output from the image pickup device per se is used as the luminance signal Y without regard to colors.
FIG. 22 shows the luminance signal Y obtained by the SWY method. In FIG. 22, suffixes indicating pixel positions are added to symbol Y representing the luminance signal. Usually, to suppress a carrier from being generated at pixel sampling in the image pickup device, a base signal is obtained by subjecting the luminance signal to horizontal and vertical low-pass filter (LPF) processing so as to make a filter-processed output zero at a Nyquist frequency.
In a case, for example, that a filter coefficient [1 2 1] is used in each of the horizontal and vertical LPF processing, a LPF-processed output Y22′ corresponding to pixel Y22 can be determined according to the following formula (2).Y22′= (Y11+2×Y12+Y13+2×Y12+4×Y22+2×Y23+Y31+2×Y32+Y33)/16  (2)
Next, a high-frequency emphasis signal is generated from the base signal and added to the base signal, whereby a high-frequency compensated signal (hereinafter, referred to as the SWY signal) can be obtained.
FIG. 23 shows resolvable spatial frequency characteristics of an OG signal and an SWY signal.
In FIG. 23, the frequency space in the horizontal direction of an object is represented along an x-axis, and the frequency space in the vertical direction is represented along a y-axis. The spatial frequency increases with increasing distance from the origin.
In the out-of-green method, since the luminance signal is generated mainly from the green signal, resolution limits of the OG signal in the horizontal and vertical directions are each equal to a Nyquist frequency (which is π/2 on each of the x and y axes). In diagonal directions, there are lines in which pixels do not exist, so that resolution frequency limits in the diagonal directions are lower than those in the horizontal and vertical directions. Accordingly, the resolvable spatial frequency is within a diamond-shaped region 2100 shown in FIG. 23.
In the SWY method, since the luminance signal is generated by using all the pixels, the resolvable spatial frequency is within a square region 2101 shown in FIG. 23 in a case that the object is achromatic. However, in the case of, e.g., a red object, since substantially no luminance signal is output from pixels other than red pixels, the resolvable spatial frequency is within a region 2102 which is one-fourth as large as the region 2101 for achromatic object.
As described above, both the out-of-green method and the SWY method have a disadvantage in respect of resolvable spatial frequency. To obviate this, there has been proposed a signal processing apparatus in which whether an image is a white-and-black image or a color image is determined, and in the case of an achromatic object, the SWY signal, instead of the OG signal, is applied to diagonal regions such as ones shown by 2103 in FIG. 23 (see, Japanese Laid-open Patent Publication No. 2009-105977).
There has also been proposed a luminance signal generation apparatus in which in the case of an achromatic object, an angle-adaptive SWY signal obtained by subjecting a signal to horizontal and vertical LPF processing and by subjecting the resultant signal to 45-degree direction LPF processing (or 135-degree direction LPF processing) is applied to diagonal regions each extending along a 45-degree line direction (or 135-degree line direction), whereas an OG signal is applied to a Nyquist region (see, Japanese Laid-open Patent Publication No. 2008-72377).
In an image pickup apparatus for photographing an image by using color filters of primary color Bayer pattern shown in FIG. 21, a sampling frequency of an image pickup device must be equal to or higher than two times a spatial frequency of an object in order to generate an accurate image. This is because there is a fear that a false color (color moire) occurs at, e.g., a bright-dark boundary of the object, if the spatial frequency of the object exceeds the Nyquist frequency (which is one-half of the sampling frequency) of the image pickup device.
The apparatus disclosed in Japanese Laid-open Patent Publication No. 2009-105977 does not consider the influence of a false color appearing at near horizontal and vertical (HV) Nyquist regions upon the determination to determine whether an image is a white-and-black image or a color image. As a result, even in the case of an achromatic object, the object is determined as being a chromatic object at the HV Nyquist regions due to the influence of a false color, and the OG signal is used. As a result, a problem is posed that aliasing occurs in the OG signal.
The apparatus disclosed in Japanese Laid-open Patent Publication No. 2008-72377 applies to the HV Nyquist regions the adaptive OG signal that limits the horizontal or vertical bandwidth in order to improve the resolution as compared to a case where the SWY signal is used. However, a problem is posed that at near the HV Nyquist regions, diagonal aliasing in the OG signal becomes large with increasing distance from the Nyquist point.